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Thermodynamics of adiabatic feedback control. 
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Abstract. - We study adaptive control of classical ergodic Hamiltonian systems, where the 
controlling parameter varies slowly in time and is influenced by system's state (feedback). An 
effective adiabatic description is obtained for slow variables of the system. A general limit on the 
feedback induced negative entropy production is uncovered. It relates the quickest negentropy 
production to fluctuations of the control Hamiltonian. The method deals efficiently with the 
entropy-information trade off. 



Introduction. Relations between the control theory and 
physics have a long history. The notorious Maxwell's de- 
mon, conceived yet in XlX'th century, is in fact a con- 
trol device that aims to reduce the entropy of a statistical 
system [1,2]. Founders of cybernetics recognized the en- 
tropy reduction as one of the basic goals of control [3-5] . 
This became even more important once it was understood 
that the statistical description and thermodynamic rela- 
tions are needed not only for the macroscopic situation, 
but also for few-body chaotic and stochastic systems [6-9] . 
Indeed, the first attempts of relating entropy and informa- 
tion during control operations were made in the early days 

\ of cybernetics [4,5] and where based on thermodynamics; 
see [1] for a fuller historical perspective. More recent the- 
ories of entropy-information-control relationship were pre- 
sented in [2,10]. The results of [10] found applications in 

\ the theory of chaotic systems control, where the entropy 
reduction is again the basic goal. This field has undergone 

\ an explosive development due to synthesing the physical 
and control scientific ideas [11-13]. 

Much attention was devoted recently to the control of 
Brownian systems (mesoscopic particles coupled to a ther- 
mal bath) [14]- [25] . This field is expected to have wide ap- 
plications in various areas of nanoscience. The first theory 
of feedback driven cooling (entropy-reduction) of a Brow- 
nian particle was developed within statistical thermody- 
namics [14]. This theory plays an important role for recent 
experimental cooling schemes in nano-physics [15,16,23], 
e.g., in atomic force microscopy [16]. In a related context, 
Ref. [20] studied experimentally how the feedback con- 
trol applied to a Brownian nanoparticle generates forces 



of rather general shape, including non-potential forces. 

Following to the experimental development of feedback 
cooling methods, Ref. [17] presented the thermodynamic 
analysis of a Brownian particle, which couples to a ther- 
mal bath and is manipulated by control fields so that to 
cool the bath. The authors of [17] gave a general recipe 
for calculating the entropy pumped out of the bath versus 
the entropy produced during the operation of the Brow- 
nian particle. The quantum extension of this setup was 
investigated in [18]. Fluctuation theorems for the classical 
Brownian control setup were studied in [19]. 

Control of Brownian particles is also employed for gen- 
erating a directed motion; see [21] for reviews. This task 
is important for contructing nanoscale engines (rachets or 
Brownian motors). Theoretical and experimental propos- 
als for feedback driven ratchets appreared recently in [22] 
and [23], respectively. 

Finally we should mention works devoted to the open- 
loop (non-feedback) control of Brownian particles [24,25]. 
These studies are especially relevant for statistical physics, 
since the basic formulations of the second law are in fact 
control-theoretical statements [26] . 

Here we explore an approach to the adaptive feedback 
control of classical Hamiltonian systems. Feedback means 
that the dynamics of a control parameter of the Hamil- 
tonian is influenced by system's state, i.e., it performs an 
adaptive motion, while in the non-feedback (open-loop) 
control the motion of the control parameter is prescribed. 
Our main assumption is that the control parameter moves 
slowly. Assuming the ergodicity of certain observables we 
develop a general thermodynamic description of the feed- 
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back control process. In particular, we find the control 
fields that ensure the quickest reduction of the entropy 
(noise). This reduction is related to fluctuations of the 
controlling part of the Hamiltonian. We also describe the 
entropy-information trade-off: how limitations of the in- 
formation available on system's state decrease the speed of 
the entropy reduction and change the qualitative features 
of the control process. 

Note that in contrast to the above works on the Brow- 
nian particles control, we shall work with the full Hamil- 
tonian system, and not with a small particle coupled to 
a large thermal bath. Moreover, we focus on Hamilto- 
nian systems with finite degrees of freedom, though the 
presented theory applies to macrocopic Hamiltonian sys- 
tems, e.g., the particle plus the bath. The macroscopic 
system control will be studied elsewhere [34]. 

Basics of the method. Consider a system with n degrees 
of freedom and Hamiltonian H(p, q, R). The equations of 
motion read 



q = d p H{p, q, R), 



P : 



d q H{p, q, R), 



(1) 



where q = (<fi, q n ) and p — (pi, ...,p n ) are, respectively, 
the coordinates and momenta, and where R is a time- 
varying parameter (the extension to several parameters is 
straightforward). The parameter R is changed externally 
controlling the system (the goals of control are indicated 
below). The evolution of R is described in the standard 
way of the adaptive control [11-13,27,28]: 



t r R = F(E,R,z), z=(q,p), \F\<F, 



(2) 



where F is the control field, tr is a characteristic time 
of R, and where E is system's energy. The constraint 
\F\ < F with constant F is a natural condition for prac- 
tical realizability of the control setup. Many control tasks 
get the real physical meaning only after imposing such 
constraints on the magnitude of the control fields [27]. 

As shown by (2), the controlling parameter R is sub- 
jected to feedback: the dynamical variables z and E influ- 
ence, via suitable engineering, the evolution of R. Control 
processes without feedback (open-loop control) correspond 
to F = F{R) 1 with predetermined motion of R. 

So far we presented a standard and rather general setup 
for control problems. In particular, Eq. (2) contains many 
adaptive control processes known in literature [11-13,27, 
28] . One of our basic assumptions is that the motion of R 
is adiabatic, i.e., that the time-scale tr of R is much larger 
than the characteristic time ts of the system (defined after 
(10)). Eqs. (1, 2) show that together with R also the 
energy E becomes a slow variable, 

^7 = ^7 9 * H ( Z > V = — F ( E > R > z ) d * H ^ R )> ( 3 ) 
at at tr 

provided that the controlling part of the Hamiltonian 8rH 
is limited. Note that ^ is equal to the small parameter up- 
times the function F(E, R, z), which changes fast together 



with z. Thus we need the constraint (2) on the magntitude 
of F(E, R, z) for the adiabatic assumption to apply. 

We want to get from (1-3) averaged equations for the 
slow variables E and R. To this end, define the micro- 
canonic distribution: 



M(z,E,R) 



S[E-H(z,R)] 



d E tt(E,R) ' 
Sl(E,R)= J Az6[E - H(z,R)}, 



(4) 
(5) 



and where S(x) and 9{x) are, respectively, the delta and 
step function. Here £l(E, R) is the phase-space volume at 
energy E; its derivative 8e^(E, R) defines the normaliza- 
tion of M(z,E,R). 

Denote by z t and R t the solutions of (1, 2). On times 
tr^> t » ts we have from (1, 2) for the slow derivative 
AE/At : 



^ ee - [H(z t+T ,R t+T )-H(z u R t )} 
At t 



(6) 



t+T dsdH 

-T-{z s ,Rs) 



I r 

ft+T 



t+T As • dH 

— -n-s -^{ z s,Rs), 

T OR 



= J_ f T ^l F{Es ,R s ,z s )^(z s ,R s ), 

TR Jt T 3R 

= ~\- T F{EuRuz s )^-{z s ,R t ) + (f ). (7) 

TR Jt T OR Tr 

As a consequence of the adiabatic assumption, the last 
integral in (7) refers to the dynamics with R t =const and 
E t =const. Denote 



w(z)=F(E t ,Rt,z)d R H(z,Rt), 



(8) 



and write the time-average in (7) as J t ^-w(z s ). Con- 
sider the following obvious relation: 

J dzw(z)M(z,E) = i J ds J dzw(z)M(z,E). (9) 

In the RHS of (9) we change the integration variable 
as y = %- s z, where T t is the flow generated by the 
Hamiltonian H{z) between times and t. Employing 
Liouville's theorem, dz = Ay, and energy conservation, 
M(z, E) = M(y, E), one gets 



(9)- J dyM(y,E)±- ^ dsw(T s _ t y). 



(10) 



If w(z) is an ergodic observable of the R t =const dynam- 
ics, then for t > T5 the time-average ^ J^ +T dsw(T s - t y) 
in (10) depends on the initial condition y only via its en- 
ergy H(y) [6,7,32]. (Thus, ts is the relaxation time of 
w(z).) In particular, the dependence of the precise value 
of y is irrelevant provided that the condition H(y) = E 
holds. Since A4(y,E) is a delta-function concentrated at 
that value of energy, the integration over y in (10) drops 
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out, and we get from (9, 10) that the time-average is equal 
to the microcanonic average at the energy E 



J dzw(z)M(z,E) = ^ J dsw(T s _ t y). (11) 
Applying this to (7) we get 

tr^ = J dzM(z,E,R)F(E,R,z)d R H(z,R),(l2) 

TR ^ = J dz M{Z ' E ' R) F{E ' R ' Z) = {F)e ' r ' (13) 

where (13) is derived analogously to (12) by assuming the 
ergodicity of F(E, R, z). 

Eqs. (12, 13) are the basic equations of the adiabatic 
feedback theory. We list again the assumptions employed 
in their derivation: i) E and R are slow variables; ii) con- 
servation of energy for R — const; Hi) Liuoville's theorem; 
iv,) ergodicity of two phase-space observables: w{z) defined 
by (8) and the controlling parameter F(E, R, z). 

Instead of 2n + 1 equations (1, 2) we have in (12, 13) 
only two equations for E and R. They are autonomous, 
since they do not depend on the precise initial value of z, 
provided it was on the initial energy shell Ei. Thus the 
control processes described by (12, 13) can operate under 
conditions, where the initial values of z are not known or 
the dependence from them is not desirable. The price to 
be paid for this is that now only functions of E and R can 
be controlled. 

Recall that for a (fully) ergodic system all the suf- 
ficiently smooth observables are ergodic, while a non- 
ergodic system can still have some ergodic observables; 
see [32] for the general theory. Now note that since no er- 
godicity of all observables is assumed in deriving Eqs. (12, 
13), they apply to some non-ergodic systems. Another rea- 
son for applying (12, 13) to non-ergodic systems is that the 
ergodicity may be restored under small perturbation [6,7]. 
Thus the scheme applies to most of chaotic systems. 

Definition of entropy. Entropy and information play 
important roles in the control theory: the very possibility 
of applying feedback is due to the information available on 
the state of the system, while entropy reduction [negen- 
tropy production] is necessary for immunizing the system 
from sources of noise and instability [2-4] . Thus our next 
task is to obtain from (12, 13) the maximal negentropy 
production rate. Recall that for a (partially) ergodic sys- 
tem the entropy is defined as [6,7]: 

S = In Q(E, R)=\nj dz 9(E - H(z, R)). (14) 

Eq. (14) satisfies to all desired features of entropy, even if 
the number n of the system degree of freedom is finite: 

1. For the temperature T defined via the standard ther- 
modynamic formula 

1/T = [3 = d E In n(E) = — J dz S(E - H(z, R)), (15) 



the integration by parts leads to equipartition [6, 7]: 
(y d y H) = T, where y is any canonical variable, while 
(...) is the average over microcanonic distribution (5). For 

2 

the standard Hamiltonian H = Y^k=i if + V{Qi> .., q n ) we 
get (pi) = T for any k, which is the standard form of 
equipartition. Note that T in (15) is non-negative. 

2. S satisfies to the first law of thermodynamics for the 
microcanonic ensemble [6,7,30]. 

3. S satisfies to two formulations of the second law that 
describe a partially ergodic system subjected to an open- 
loop (i.e., non- feedback) control: i) S remains invariant 
in an adiabatically slow [open-loop] process; see [6-9] and 
the discussion after (17). ii) S increases under a non- 
slow [open-loop] process, provided that the system starts 
its evolution from an equilibrium state [8,31]. The lat- 
ter formulation is closely related to the minimum work 
principle [26]. 

In the thermodynamical limit S goes to the more usual 
Boltzmann expression [6,7]: 



S B (E) = ln[d E Q] = Jdz 5{E-H(z,R)). 



(16) 



None of the above features 1-3 holds if we apply Sb out 
of the thermodynamic limit, e.g., Sb is not constant for 
open-loop adiabatic processes. This is because S is the 
unique adiabatic invariant for ergodic systems; see [6, 7] 
and the discussion after (17). Another problem with using 
the Boltzmann expression Sb directly for finite systems is 
that the temperature defined via (16) and the standard 
thermodynamic formula as 1/Tg = 8eSb(E) is in general 
not even positive [7]. The problems in attempting to use 
Sb as the proper entropy will be illustrated below by a 
concrete example; see the discussion after (34). 

We are thus convinced that S is the proper expression of 
entropy for both finite and macroscopic ergodic systems. 

Negentropy production. We now determine the evolu- 
tion of the entropy S according to (12, 13). Using (12-15) 
we get 

TR ^-=f3(E,R)(F [d R H-(d R H) EtR ]} E!R , (17) 

where (...) e .r is defined in (13), and where [3(E,R) > 
is the inverse temperature defined in (15). Eq. (17) 
shows that if there is no feedback over the fast variable 
z, F — F(E,R), the entropy S = lnCl(E,R) is conserved 
on the slow time, i.e., it is an adiabatic invariant [6-9]. 
Recall that for ergodic systems this is the unique adiabatic 
invariant: any other quantity K = K{E 1 R) that remains 
constant for open-loop adiabatic processes is a function of 
the phase-space volume 0: K = K(£Y) [6,7]. Indeed, since 
in general d E Q,(E, R) > we can express (for a fixed R) 
E as a function of f2 and R: E = E(Q, R). Putting this 
into K(E, R) we re-express it as a function of f2 and R: 
K = K(Cl, R). Differentiating K(fl, R) over the slow time 
we get: 

AK _ dfl dK dR dJX 

d7 ~ "d7 W + "d7 dR' ( ' 
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Since both and K are assumed to be adiabatic invari- 
ants, = ^ = 0, we get from (18) that §f = 0, i.e., 
K is a function of f2 only. 

However, for feedback processes the entropy docs 
change. Let us find the feedback F(E, R, z) that maxi- 
mizes the negentropy production —^7 under the natural 
constraint |F| < F. Since the RHS of (17) is a linear func- 
tion of F, and since (3 > 0, the extremum is achieved on 
the boundaries |F| = F of the allowed range. This implies 
for the most negative negentropy production which 

we denote as 

T R ^; = -f3F(\d R H-(d R H) EiR \} EtR , (19) 

F(E,R,z) = -Fsign[d R H-{d R H) E , H ]. (20) 

Eq. (19) bounds the rate of the entropy reduction, and it is 
related to fluctuations of the control Hamiltonian d R H on 
the surface of constant energy. The optimal control func- 
tion (20) is seen to be discontinuous. Note that Ref. [25] 
reports that discontinuous control fields is a general fea- 
ture of the optimal open-loop control that operates in a 
finite time. We see a similar effect for a feedback setup, 
which is not constrained globally to a finite operation time. 

The phase-space volume Q(E, R) is a Lyapunov func- 
tion for the dynamics described by (12, 13, 20), since it 
is obviously non-negative, and since it is non-increasing, 

T R^£P < 0j as follows from (19). The non-negativity 

and non-increasing of ^ imply, via the Lasalle princi- 
ple [29] i Jhat the long-/ solutions E, R of (12, 13, 20) 

satisfy ^(E, R) = 0, which leads via (19) to 



/ 



dzS(E-H(z,R))[d R H(z,R)-{d R H} EtR }=0. (21) 



There are two possibilities for satisfying the equality in 
(21): i) the long-r solutions converge to a stable fixed 
point (i.e., energy minimum) of the original Hamiltonian 
system (1). This means that the phase-space volume 
fl(E,R) decays to zero, while the entropy \n£l(E, R) de- 
cays to its minimal value minus infinity, ii) The second 
possibility of realizing (21) is that the long-T solutions con- 
verge to a point (E, R) such that d R H(z, R) as a function 
of z is constant on the energy shell H(z,R) — E, i.e., it 
is a constant of motion for the fixed values of E = E and 
R = R. Since the second option is unstable to small per- 
turbation in the control Hamiltonian d R H(z, R), the first 
option is more likely to be realized: the feedback setup 
(20) drives the system toward an energy minimum, where 
the phase-space volume ft is equal to zero. 

Limits on the available information. When obtaining 
the maximum rate (19) of negentropy production we only 
assumed that the magnitude |F| of the controlling parame- 
ter is limited from above; see (2). More crucial restrictions 
come into play when noting that in practice the very infor- 
mation available on the state of the system is limited. We 



thus assume that the full knowledge of z is not available 
for the controller; only some function $>(z) of z is known, 
and thus the feedback F in (2) depends on z only via <j>(z): 



F(E,R,z) = f(E,R,<t>(z)). 



(22) 



Note that this implies a reduction of the data z, 
since for simplicity we take one — in general not one- 
to-one — function $(z) instead of the vector z = 
(qi, ...,q n ;pi, ...,p n ). This is the standard way of mod- 
eling the data reduction in the information theory, known 
also as coarse-graining or statistics taking [33]. All the 
standard measures of information — e.g., Shannon entropy, 
relative entropy, the Fisher information — are known to de- 
crease after taking a (not one-to-one) function of the data. 
In other words, the data reduction means information de- 
crease with respect to any definition of information. In 
the extreme case, where no information is available for 
the feedback we have <j>(z) = const. Rewriting (17) as 

T R —=p(E,R) J dyf(E,R,y)^(E,R,y), (23) 

where we defined 

ip(E,R,y) = J dzM{z,E,R)x (24) 

6[y - $(*)] [d R H(z,R) - (d R H) EtR ], 

and applying the same derivation as for (20, 19), we get 
for the most negative ^ at the given $(z): 



dS _ _{3F_ 
dr t r 



dy\tl>{E,R,y)\. 



This value of 4^ is achieved for the feedback: 



f(E,R,y) = -Fsign[iP(E,R,y)}, 



(25) 



(26) 



where F(E,R,z) is recovered from (22, 26). Thus the 
maximal negentropy production under information limi- 
tation is related to fluctuations of the control Hamilto- 
nian d R H over a constrained microcanonic ensemble; see 
(24, 25). As follows from (17, 25), the speed of entropy 
reduction decreases under information limitations: 



(27) 



In particular, tp = 4^- = for <f> = const. 

Examples. We illustrate the obtained feedback schemes 
via the celebrated example of adiabatic physics that is a 
harmonic oscillator with the controlling frequency: 



H = Y + — 



(28) 



This Hamiltonian with the feedback controlling frequency 
is close to the experimental situation realized in Ref. [20] . 
In the context of adiabatic feedback control, the harmonic 
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Fig. 1: Entropy reduction scheme (33, 34) for harmonic oscil- 
lator. Slow variables versus dimensionless time t/tr: Energy 
E (lower curve), frequency R (upper curve), and entropy S 
(dashed curve). For the parameters we take F = 1 and L = 1. 
The curves saturate for t/tr > 2.5 when the available infor- 
mation is not anymore sufficient for any entropy reduction. 



oscillator (28) displays two interesting effects that exist 
as well in more elaborated situations [34]: control with- 
out systematic motion of the controlling parameter and 
qualitative changes in the control setup upon information 
limitations. For a constant R the period of the oscillator 
is 2n/y/R, and the adiabatic approach applies at least for 
1 <C tr^/R/2-k. This is an crgodic system and Eq. (20) 
implies for the optimal entropy-reducing control 



F(E,R,q) 



F E 
— signer 2 - 



(29) 



Eq. (29) leads via (12, 13, 5) to R constant in the slow 
time (though R is not constant on the fast time), 



dR 

7T7 



= 0, 



exponential decay of energy (cooling) , 



E(t) = E(0)e~ 



ttR ' 



(30) 



(31) 



and thus to linear decay of the entropy S = \n[^=\: 

S{t) = 5*(0) — ^ As intuitively expected, entropy 
reduction relates to cooling. Eq. (30) shows that the con- 
trolling parameter R does not move in average, i.e., on the 
slow time. The cooling is achieved due to the motion of R 
on the fast time-scale; see (2, 29). 

To limit the information available about the coordinate 
q, we assume that it is only known whether q is larger 
or smaller than a positive constant L. The function $(q) 
in (22) thus takes only two distinct values, e.g., <J>(q) = 
0(L — q). This brings from (26, 22) a control setup 



F(E, R,q)=F sign(L - q) 9(2E - RL 2 



(32) 



The latter step function is there, since for small E (or 
large R) the oscillator is located next to q = and its 
position cannot be utilized by the feedback. Thus for those 
values of E it is best to do nothing, F = 0, since any 



control (under assumed information limits) will increase 
the entropy (disorder). We get from (12, 13, 32) and from 
(22-25) 



dR 
7P7 



2F 
trit 



(1 — £) arcsin£, £ = 



RL 2 



2E ' 



dS d , r 2nE, 



F 



ttRt r 



(33) 

o(i-oaV±~e, (34) 



while the equation for 4^ can be recovered from (33, 34). 
The behavior of E(t), R{t) and S(t) is shown in Fig. 
1. We see that the entropy reduction rate is not simply 
smaller than the optimal one, but it is realized via energy 
increase (heating) rather than cooling. 

For the considered oscillator model, let us illustrate that 
the Boltzmann expression is not the proper defintion of 
entropy for a finite system. Recalling (14, 16) and using 
(34) we get that for the harmonic oscillator (28): Sb = 
ln[-^2=]. It is seen that i) Sb does not depend on the 
energy, so that attempting to define the temperature via 
the standard formula (15) will lead to zero temperature, 
not a reasonable result, ii) Sb is not adiabatic invariant, 
one can decrease it via an open-loop control by changing 
R. 

Adiabatic invariant. Eq. (32) provides a control setup, 
where the dependence on the slow and fast variables fac- 
torizes: 



F(E,R,z) = g(E,R)cj>(z). 



(35) 



For this case the slow variable system (12, 13) possesses 
an integral of motion, i.e., an adiabatic invariant. One 
deduces from (12, 13): 



^- J dz 4>{z) 9{E - H(z, R)) = 0. 



(36) 



This conservation generalizes to many-dimensional sys- 
tems the adiabatic invariancc found in [35]. For <j){z) = 
const, where there is no feedback over fast variables, we 
are naturally back to the usual conservation of the phase- 
space volume. 

In conclusion, we developed an adiabatic approach for 
the adaptive feedback control of Hamiltonian systems. It 
is derived assuming ergodicity of two observables and thus 
applies to the most of chaotic systems and some inte- 
grate ones. The approach reduces the control problem 
to two equations (12, 13) describing the evolution of slow 
variables. With help of these equations we got a general 
upper bound (19) on the rate of negentropy (order) pro- 
duction induced by the feedback control. This bound is 
achieved for discontinuous control field (20), and is related 
to the fluctuation of the control Hamiltonian over the mi- 
crocanonic ensemble. 

The method describes the information-entropy trade- 
off: how the maximal negentropy production rate de- 
creases when the information available for the feedback 
gets limited. The example of harmonic oscillator with the 
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controlling frequency shows that information limitations 
do change the quanlitative features of the control dynam- 
ics. In particular, the entropy reduction is realized via 
heating the system. Note that in the present approach 
we standardly modeled the information limitation via the 
reduction of the data available to the controller. 

The Hamiltonian dynamics finds applications well be- 
yond the proper (statistical) mechanics, e.g., in hydrody- 
namics [7] or in ecology [36] . Control issues in these fields 
are well known, and since our methods are not system- 
specific, they may apply to controlling a vortex flow, or to 
optimizing the harvest production [34]. 

* * * 
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